Computer vision, otherwise referred to as image processing, involves the extraction of vision-related information by obtaining signals representing a scene and performing image signal processing on those signals. Applications of computer vision (image processing) techniques include character recognition, industrial inspection of manufactured items, robot guidance systems, radiology, remote sensing, and so on.
Image processing methods may typically comprise several common processing steps, e.g., as described by Rafael C. Gonzalez and Paul White in a book entitled "Digital Image Processing," Addison-Wesley (1992), the content of which is hereby incorporated herein by reference in its entirety.
In a first step, image acquisition is performed to acquire an image of the scene to be analyzed. The image may, for example, be represented in the form of a monochrome or simple digital image f(x,y) discretized both in spatial coordinates x,y and in brightness (gray levels) f.
Pre-processing and enhancement techniques may then be performed on the digital image in order to improve the image and increase the chances for success in subsequent processes. Such pre-processing and image enhancement techniques may include enhancing the contrast between one or more object images and a background image, and filtering noise from the image.
Segmentation may then be performed, which involves distinguishing between different types of regions within the input image, e.g., distinguishing between an object image and a background image.
In a next step, representation and description processing may be performed. In performing representation processing, the data obtained as a result of segmentation is converted into a "representation" suitable for computer processing. In performing description processing, "feature descriptions" are extracted from the representation. Such "feature descriptions" may categorize objects to allow one class of object to be differentiated from another.
In a final stage of the image processing, recognition and interpretation processes are performed. Recognition processing may include assigning a label to an object based upon information provided by its "feature descriptions." Interpretation processing may assign a meaning to an arrangement of recognized objects. For example, several characters may be "interpreted" as forming a particular word.
Golden Template Comparison (GTC) is an image processing method that has been used to detect flaws and defects in two-dimensional scenes of an inspected item which are highly repeatable and do not suffer from geometric distortion. Such scenes are common in semiconductor production and graphic arts applications. Generally, GTC involves the comparison of a test image to an image of a known good scene referred to as a template image. More specifically, the test image is subtracted from the template image, and differences between the two images are observed in order to determine whether a flaw or defect is present within the inspected item.
Generally, GTC comprises two main phases: training and inspection. During training, the template image is constructed by sampling a plurality of images each of which represents a scene of an item absent defects. During inspection, the test image, which represents the scene of the item to be tested (the inspected item), is then compared to the template image. More specifically, the images are subtracted to form a difference image. Thresholding is performed on the difference image to produce a resulting binary image called an error image containing either background (healthy) pixels or defect pixels which represent flaws. The error image is then analyzed in order to determine whether the tested item should be labeled as an "accept" or "fail" item. The analysis that may be performed on the error image may include counting defect pixels found in the error image and performing a blob analysis on the defect pixels. The results of the analysis may be stored in a data structure called a results structure.
A. Issues Concerning the Use of Golden Template Comparison to Inspect the Inner Side-Wall of a Labeled Bottle
(1) Changes In Image Intensity
In order to produce (with thresholding) an error image which effectively and accurately represents defect pixels separate from healthy pixels, pixels within the difference image must have a sufficiently high gray level value. In order for this to occur, defects must give rise to a change in image intensity in the test image. In graphic arts applications, defects are typically defined in terms of changes in scene reflectivity, which result in a change in image intensity in the test image. Semiconductors are inspected to determine if they contain surface contaminants that can result in a subsequent device failure. Such surface contaminants frequently cause changes in scene reflectivity which thereby change the image intensity of the test image.
In each of these situations, the defect is externally-visible. A problem arises, however, when inspecting semi-opaque enclosures (e.g., when inspecting the inner side-wall of a labeled bottle) to determine whether they contain unwanted objects (defects). In that application, the defect is not externally visible. Therefore, when inspecting the inside of semi-opaque enclosures for defects (non-conforming objects), it is difficult to obtain a test image that exhibits a change in image intensity due to the presence of a defect.
(2) Gray-level Defect Criteria
In order to produce an error image which accurately distinguishes defect pixels from background (healthy) pixels, thresholding may be performed in accordance with appropriate gray-level defect criteria. Various thresholding methods have been proposed which perform thresholding in accordance with different criteria. A simple type of defect criteria is photometric defect criteria which classifies pixels as background (healthy) pixels or defect pixels based upon image intensity alone. Other defect criteria (e.g., based on the shape formed by a group of adjoining defect pixels) can be utilized to guide the thresholding process.
(3) Geometric and Morphological Defect Criteria
The binary error image may be further processed to correct (replace) improperly classified pixels. For example, the binary error image may comprise defect pixels falsely classifying healthy pixels as defects. Accordingly, the binary image may be processed by imposing additional criteria, e.g., based, upon the size and/or shape of a set of adjoining defect pixels. Simply put, defect pixels formed in the error image from small intensity variations in the test image may be ignored (i.e., replaced with healthy pixels) unless they represent a defect which is of a certain size and/or shape (e.g., if the defect is at least several pixels in length or diameter). Geometric and morphological defect criteria are disclosed by William M. Silver and Jean Pierre Schott in a document entitled "Practical Golden Template Comparison," provided by Cognex Corporation, the content of which is hereby incorporated herein by reference in its entirety.
The usefulness of defect criteria (such as geometric and morphological defect criteria) can be significantly diminished when a distorted test image is acquired, e.g., by a conventional bottle inspection arrangement, as shown in FIGS. 7A-7C.
A conventional labeled-bottle visual inspection system is shown in FIG. 7A which comprises a vertically-arranged, elevated camera 200 having a wide-angle lens 200a pointed directly into an upper opening of a bottle 202. Bottle 202 has a semi-opaque label 203 surrounding its outer side-wall surface 210, and is positioned over back lighting 206. A defect 204 is depicted in FIG. 7A with dotted lines, because defect 204 is not visible from outside of bottle 202 due to the opaque nature of label 203. Defect 204 is on an inner side-wall surface 208 of bottle 202, behind label 203. FIG. 7B comprises a cut-away view of bottle 202, in which inner side-wall surface 208 is exposed, and defect 204 is viewable. The view provided in FIG. 7B directly faces defect 204. As depicted, defect 204 is dark in appearance and rectangular in shape.
FIG. 7C shows a 2D test image 201 which is acquired by camera 200 with the use of a wide-angle lens 200a. In order for the complete inner side-wall surface 208 which spans the entire area of label 203 to be inspected, wide-angle lens 200a is used which results in a distorted test image 201. The distorted nature of 2D test image 201 is evident from the relative positions of various portions of bottle 201 within the test image, including lower rim 212 of bottle 202, bottom edge 216 of label 203 and top edge 214 of label 203. The entire area of interest lies between bottom edge 216 and top edge 214.
As shown in FIG. 7C, test image 201 includes a distorted depiction of defect 204. Due to the significant distortion of the shape and size of defect 204 in the 2D test image 201 shown in FIG. 7C, it is difficult to define defect criteria which can be used to accurately identify the existence, size and position of defects. The above-described geometric and morphological defect criteria are not sufficient for this purpose.
(4) Training The Template Image
In performing training, a data structure is created containing information important for the subsequent inspection of a test image. The process includes acquiring several sample images taken under varying conditions, and then calculating statistics from data gathered in connection with the sample images. The calculated statistics include a template image which is the arithmetic mean of the several sample images. The template image is the image which is subtracted from the test image, and may be further used for purposes of normalization mapping. The calculated statistics also include a standard deviation image. This image may be used to derive a threshold image, which may be a linear mapping of the standard deviation image. Other statistics may be calculated from the acquired sample images for purposes of calculating a normalization map. Such statistics may include a histogram, mean, standard deviation, and left and right tails.
The training that is required in Golden Template Comparison is both complicated and time consuming.
(5) Repeatability of Test Image
In addition, in order for the information obtained during training to be useful, the sample images must be highly repeatable (i.e., almost identical in visual appearance) and the 2D test image must not be distorted, e.g., rotated, scaled, or skewed.
These conditions cannot be met when inspecting for defects (or other non-conforming objects) within a semi-opaque enclosure such as a labeled bottle. In the conventional labeled bottle inspection system illustrated in FIGS. 7A-7C, the defect 204 is on an inner side-wall surface 208 of a bottle 202, hidden behind a label 203. Sample images, which would necessarily include an image of label 203, would not be repeatable. This is because the shape of label 203, which forms part of 2D test image 201, changes from bottle to bottle, due to typical variations caused when label 203 is applied to the outer side-wall surface 210 of bottle 203. In addition, 2D test image 201 is clearly distorted due to the use of a wide-angle lens 200a in acquiring the test image.